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Description 

FIELD OF THE INVENTION 

This disclosure reiates to neurocomputer systems which simulate the nervous systems In living structures, 
and which provide capabilities of solving complex combinatorial problenris in a relatively rapid and economic 
fashion. 

BACKGROUND OF THE INVENTION 

Considerable studies and research for a number of years has continued in the functions of living neurons 
which are grouped in complex interconnected networks and which can be seen to act as types of problem- 
solving computer systems. 

By posing electronic networks which simulate the interconnected neuronic networks, it has been felt that 
manv problems could be efficiently solved through the computational capabilities required for solving a dass 
of problems which involve "combinatorial complexity". 

This dass of problems are often found in engineering and commercial ventures from simulation of the prob- 
lems of perception and recognition which must be handled by the nervous systems of living creatures. 

For example, if one is given a map and has the problem of driving from a given source dty over to a des- 
tination city, which could be considered as the best route to travel? Another common example is the problem 
of designing circuit boards in regards to where is the best location to put each of the multiple numbers of chips 
in order that an efficient wiring layout can accompany the components. ^ 

These types of analogies and optimizational problems have been described in the publication Biological 
Cybernetics by J. J. Hopfield and D. W. Tank, published 1985 in Volume 52 at pages 141-152. This artide is 
entitled "Neural Computation of Decisions in Optimization Problems" 

Studies of neural architecture indicate that a broad category of parallel organization is in operation rather 
than a series of step-by-step processing functions as occurs in general purpose computers. If each of the neu- 
rons is considered as a simple processing unit, it is seen that these biological systems operate in a "collective" 
or a "group operative" mode with each individual neuron connected to and summing the inputs of hundreds of 
other individual neurons in order to determine its own output signal. 

The computational load of taking these many sensory inputs and reducing them to a "good" or even an 
"optimum" solution seems to be one feature of these biological and neuronic system networks. Apparently, a 
collective solution is computed on the basis of the simultaneous reactions, on each other, of hundreds of neu- 
rons (or processing units which simulate them). 

Thus, if a multiple number of interconnected neurons (or processors) are fed input signals, each of the in- 
dividual neurons will provide an output signal at some time according tojts own individual input signal and to 
those parallel interconnected impulses it receives from its neighboring neurons (processors). The overall result 
of the output signals of the collective group of neurons is a "global output signal" which represents the collective 
judgement of the neural network. 

Modern digital general purpose computers using the standard Very Large Scale Integrated circuitry will 
generally involve logic gates where each logic gate will obtain inputs from two or three other gates in order to 
come to certain binary decisions in the course of a computation. However, in the situation of non-linear neural 
processors (neurons), organized in a collected parallel-processing network, these get inputs from practically 
all of the other neural processors and then compute a "collective" solution on the basis of simultaneous inter- 
actions of the hundreds of units or devices involved. 

The neural computational network will be seen to have three major forms of parallel organizations: (i) par- 
allel input channels, (ii) parallel output channels, and (iii) a large amount of interconnectivity between the proc- 
essing elements (neurons). 

The processing components are considered to have "time constants" which provide for the integrated sunrv 
mation of the synaptic input currents from other neurons in the network. In simulation of the biological neuronic 
network, it is possible to determine the interconnection paths so that they will be of an (i) excitatory or influential 
nature and (it) an inhibitory or negative, suppressive nature. 

J. J. Hopfield of the California Institute of Technology has shown that the equations of motion for a network 
(with symmetric connections) will always lead to a "convergence" of "stable states" in which the outputs of all 
neurons remain constant. 

Thus, networks of neurons with this basic organization can be used to compute solutions to specific op- 
timization problems by first choosing connectivities and input bias currents which appropriately represent the 
function to be minimized. 
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After the programming of the network is organized, an initial set of Input voltages are provided and the 
system then converges to a "stable state* which minimizes the function. The final "stable state* is Interpreted 
as a solution to the problem. Thus the set of outputs then provides an answer which is considered to represent 
a solution to the problem. 

5 An example of such a combinatorially complex problem is often cited as the TSP or Travelling Sal^man 

Problem where, for example, it Is given that a salesman is to travel to N different cities and then to find but 
the optimal tour or travel sequence necessary to minimize time and costs. 

This type of problem which might nonmally. involve "N-factorial* numbers of computations using ordinary 
computer networks, is found to come to relative rapid and efficient optimization solutions using the neural par- 
10 aliel-cost-related biological type architecture. 

A network organized neuronicaily for the solution of the TraveDing Salesman Problem may be referred to 
as the TSP network." Then, to enable the N neurons in the TSP network to compute a solution to the optinrv 
izatton problem, the network is described by an energy function (E) in which the lowest energy state (consid- 
ered the most stable state of the network) is taken to conrespond to the best path or tour. 
15 The concept of "convergence* is used to indicate that the network has settled on a final set of condition- 

states in each of the neurons (processors) and these states are no longer changing. 

Thus, system networks of microelectronic neurons (simulating biological neuronic systems) would appear 
to rapidly solve difficult optimization problems by the system of "convergence" whereby the states of the neu- 
ronic processors involved, have settled down to a "non-changing" minimal energy (E) state. 
20 These systems appear to be uniquely adaptable to the handling of combinatorial optimization problems 

which involve finding the minimum value of a given function which nomially depends on many parameters. 

An article in the publication Science on May 13, 1983, in Volume 220, pages 671 through 679. and entitled 
"Optimization by Simulation Annealing" has compared the analogy of annealing solids as providing a frame- 
work for optimization of the properties of large and complex systems. 
25 The subject of combinatorial optimization involves a set of problems which are central to disciplines of conv 

puter science. Research in this area aims at developing efficient techniques for finding minimum or maximum 
values of a function having many Independent variables. The TSP problem t>elongs to this area of research 
and is often used and tested for results using experimental procedures. 

As indicated by the authors Kirkpatrick, Gelatt, Jr., and Vecchi In the Science article, there are two basic 
30 Strategies in the heuristic methods of problem solving. The first may be called (i) "divide-and-conquer" and the 
second (ii) may be called "iterative improvement". 

In the first strategy, (i). one divides the problems into subproblems of manageable size, then solves the 
subproblems. then the solutions to each of the subproblems must be patched together in order to produce an 
overall solution. 

35 In the second strategy, (ii). of iterative improvement, one starts with the system in a known configuration 

and then a standard "reanrangement operation" is applied to all parts of the system, in turn, until a rearranged 
configuration that improves the cost or energy function (E) is discovered. The "rearranged configuration" be- 
comes the new configuration of the system and the process is continued until no further improvements can 
be found. 

40 It may be noted in the strategy (ii) that this search may sometime get stuck in a local minima" which is 

not yet a "global optimum", and thus it is customary to carry out this process several times while initially starting 
from different, randomly selected configurations, and then to go ahead and save the best result. 

Condensed matter physics is a body of methods for analyzing aggregate properties of the large number 
of atoms to be found in samples of liquid or solid matter. This physics uses statistical mechanics as a central 

45 discipline. 

Because of the tremendous numbers involved, such as, for example, wherein the numt>er of atoms involved 
is on the order of 1023 per cubic centimeter, then only the "most probable" behavior of the system in thermal 
equilibrium, at a given temperature, is observed in experiments. 

Afundamental question concerns what happens to this type of system in the limit of low temperature— for 
50 example, whether the atoms will remain fluid or will solidify, and if they solidify, will they form a crystalline 
solid or a glass. 

Experiments that determine the low-temperature state of a material, for example, by growing a single crys- 
tal from a melt— are done by slow, careful annealing. This is done by first melting the substance, then lowering 
the temperature slowly, and spending a long time at temperatures in the vicinity of the freezing point 
55 This is comparable to the previously mentioned condition of lowering the energy level (E) in order to find 

the optimum condition of the network. 

Finding the low-temperature state of a system when a prescription for calculating its energy (E) is given, 
simulates an optimization problem which is not unlike those encountered in combinatorial optimization. 
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When applied to the TSP, the •cost function" can be looked at as playing the role of the energy state (E). 

Using the "cost function" in place of the energy (E> and defining the configurations by a set of parameters, 
it is common to use a developed algorithm to generate a population of configurations of a given optimization 
problem at some effective temperature. This temperature is simply a control parameter in the same units as 
5 Is the cost function. 

Now, the "simulated annealing process" consists of: 

(1) Melting the system being optimized at a high effective temperature: 

(2) Lowering the temperature by slow stages until the system freezes": 

(3) Noticing that no further changes are occurring. 

10 At each temperature, the simulation must proceed long enough for the system to reach a "steady state". 

The sequence of temperatures and the number of rearrangements of the parameters attempting to reach equi- 
librium at each given temperature, can be considered an "annealing schedule". This type of schedule can be 
applied to an electronic network which simulated neuronic networks. 

The use of parallel networks having different levels of "connection strengths" between each of the net- 

15 works is discussed in an article, "Boltzmann Machines: Constraint Satisfaction and Networks that Learn", pub- 
lished in May 1984 by authors Hinton. Sejnowski. and Ackley through the offices of the Department of Conv 
puter Science, Carnegie-Mellon University and designated as technical report CNU-CS-84-119, and which was 
later published in the magazine Cognitive Science , Volume 9, 1985, pages 146-169 under the title of "A Learn- 
ing Algorithm for Boltzmann Machines". This involved thestudy of "connectionisf systems that store their long- 

20 term knowledge as the strengths of the connections between simple neuron-like processing elements. These 
networks are apparently suited to tasks like visual perception which can be performed efficiently in parallel 
networks and which have physical connections in just the places where processes need to communicate. In- 
cluded In the Technical Report were the observations on a parallel constraint satisfaction network called a 
"Boltzmann Machine" which is apparently capable of learning the underlying constraints that characterize a 

25 "domain of information" simply by being shown examples of information from the particular domain. The net- 
work modifies the strength of its "connections" so as to construct an internal generative model that produces 
examples with the same probability distribution as the examples it is shown. 

The Boltzmann Machine is composed of computing elements called "units** that are connected to each 
other by "bidirectional links". A unit is always in one of two states-"on" "ofr— and it adopts these states as a 

30 probabilistic function of the states of its neighboring units on the N "weights" on its links to them. These 
"weights" can take on real values of either sign. 

A "unit" being "on" or "ofr is taken to mean that the system currently accepts or rejects some elemental 
hypothesis about the domain of information. The "weight" on a link represents a weak pairwise constraint be- 
tween two hypotheses. A "positive weight" indicate that the two hypotheses tend to support one another, if 

35 one is currently accepted, then accepting the other should be more likely. Conversely, a "negative weight" sug- 
gests that, other things being equal, then the two hypotheses should not both be accepted. 

One apparent aspect of the Boltzmann Machine arrangement is that it leads to a domain-independent 
learning algorithm that modifies the connection strengths between units in such a way that the whole network 
develops an internal model which captures the underlying structure of its environment 

40 The learning algorithm presupposes that the network reaches "thermal equilibrium" and that it uses the 

co-current statistics, measured at equilibrium, to create an energy landscape that models the structure of the 
ensemble of vectors produced by the environment At the same time it should be noted that there is nothing 
in this learning algorithm to prevent it from creating an energy landscape that contains large energy barriers 
which then prevent the network from reaching equilibrium. 

45 Neurons are recognized as complex biochemical entities and it is not considered that these simple binary 

units, such as the Boltzmann Machine system, are a full representation or simulation of the actual models of 
neuronic networks. However, the assumption is used that the "binary units" change state "asynchronously" 
and that they use a probabilistic decision rule. 

The "energy gap unit" for such a "binary unit" seems to play a role similar to that played by membrane 

50 potential for a neuron. Both of these are the sum of the excitatory and inhibitory inputs and both are used to 
determine the output state. 

The "energy gap" represents the summed output from all the recent active binary units. If the average time 
between updates is identified with the average duration of the neuron's "post-synaptic potential", then the bi- 
nary pulse between updates can be considered to be an approximation to the post-synaptic potential. The sum 

55 of large number of stochastic pulses is independent of the shape of the individual pulses and depends only on 
their amplitudes and durations. Thus, large networks can act to provide the "fan-in" effect which nr^y be typical 
of the average cerebral cortex. 

"Random asymmetries" or "noise" in the system would appear to be reduced through the hierarchical struc- 
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ture providing the "fan- in". 

It is also considered that there are certain effects in the biological nervous system called •time-delays", 
but that these be considered to act like added "noise". It Is considered that the two main Ideas that led to the 
Boltzmann Machine are that "noise" can help with the searching corhputation process: and that Boltzmann 
5 distributions make it possible to assign credit on the basts of "local" information In a non-linear network. 

The Hopf ield patent US-A-4,660,166 discloses a network for amplifiers, connected to a matrix of Input and 
output conductors. Each connection is Implemented with resistors Rg = Rp between the output of amplifier j 
and amplifier i. The resistance Rg = Rp Is selected to have a value that - with appropriate signals at all input 
conductors - the network will collectively drive to a stable state at the complementary output terminals which 
10 provide an output code word that is a very good solution to the problem. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a global system view of a plurality (N) of nodal processors organized in nodal (nodes) groups all 
IS connected to a network history memory. 

FIG. 2. Is a schematic drawing of a 16-processor (neurons) global network which Is illustrative of the solving 
of the TSP, Travelling Salesman Problem. 

FIG. 3 is a schematic example of a number of groups of nodal processors shown to be interconnected each 
to another, with f bced connection weights (delay times) whereby connection weights are organized as positive 
20 (excitatory) or as negative (inhibitory). 

FIG. 4 is a schematic drawing of the network history memory which carries the state history (1 = "on" and 
0 = "off") of the past status states of each of the nodal processors for each of the past machine cycles. 

RG. 5 Is a typical output trace that might be shown for op^tjmization of the TSP, Tra^ Salesman Prob- 
lem, for a tour of six cities. 

25 FIG. 6 is a schematic representation of the nodal weight circuits which indicates the subject of the con- 

nection "weight" between each and every one of nodal processors In the network. 

HG. 7 is a drawing of the energy state of a Global Network used for experimentation showing the energy- 
minimization with time as greater optimizing solutions are executed. 

FIGs. 8 and 9 are histograms which show the probability of a neuron (processor) changing its output state 
30 as a result of the "weighted sum" of all the inputs (5E) to the neuron (processor) system network. 

GENERAL OVERVIEW 

"Simulated annealing" is an efficient, maximum entropy algorithm for solving nonpolynomial time (NP) 
35 complete optimization tasks. By judiciously controlling "noise", it mimics the natural process of annealing, 
where the temperature (T) is associated with "noise" and the objective function of "stability" is associated with 
the physical energy state. 

Disclosed herein, as claimed, is a network of binary neuronal processors provided with "fixed" connection 
delays. Because of the fixed connection delays between and every one of the neuronal processors, the net- 

40 work "cools" as It approaches the condition of stability. This seems to be a curiously inherent feature of such 
a "delayed" model system. 

As will be observed in FIG. 7. there can be seen the minimization of the network's "energy" (E) which oc- 
curs as a function of time. Through the use of the Veight"of the interconnections between each of the neuronal 
processors, the energy (E) can be mapped to fit the objective function of a variety of optimization tasks. 

45 Previous investigators have shown that an analog realization of such a neuronic network can be used to 

solve a variety of optimization tasks. However, this disclosure indicates that an analog representation is not 
necessary and that simple binary processors, using connection-delays will provide rapid solutions to optimi- 
zation tasks. The system described here would represent a simple and fast special purpose parallel processor 
system, and since this network uses only binary processors, it can be implemented simply in traditional very 

50 large-scale integration circuitry (VLSI). However, other more complex implementations are also feasible. 

In regard to the previously mentioned TSP. or Travelling Salesman Problem, reference may be made to 
FIG. 2 which involves a system of 16 processors which might be used for solving a four-city TSP. As seen in 
FIG. 2, there are 16 processors organized in groupings of four processors each. These groupings of four proc- 
essors are designated with I. II, III. IV. Within each group there are set of four processors each having a nu- 

55 merical designation; The TSP is considered as the archetype of typical hard optimization problems, and can 
be stated as: Find that sequence of routings which will minimize the distance travelled by a travelling salesman 
who must travel to N cities stopping at each city only once. Legitimate travel tours can be represented as a 
permutation of N integers where each city Is labeled with a number. Thus in each of the four groupings of four 
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processors, the processor No. 3 would represent city No. 3, while processor No. 2 woutd represent city No. 2, 
and processor No. 1 would represent city No. 1 , etc. 

At the bottom of FIG. 2. there Is shown a sequence of "states" of at least one processor from each one of 
the four groups. Thus the sequential symtjols 3, 2, 4, 1 might represent one possible tour having the meaning 
of starting at city 3, then proceeding from city 3 over to dty 2, then proceeding from city 2 over to city 4, then 
proceeding from city 4 over to city 1 . 

Each of the 16 processors shown in FIG. 2 are binary types which means their output is either "on" or it 
is "off." Those processors which currently have an output as "on" are shown in the heavy black lines which 
are completely filled in while all those processors whose output is "ofr have empty white space within the per- 
iphery of their circles. 

Thus the hardware for solving the four-city problem might consist of four groups of four processors each. 
Each group (I or II or III or IV) of processors represents a place in the permutation. Thus within each group, 
each processor is identified with that number which represents a particular one of the four cities. Each proc- 
essor can be in only one of two states— "on"— meaning that this city is the one that should take this place or 
position in the permutation, or "ofr, the reverse. The connections between each of these processors are chos- 
en to encourage the entire network of processors to choose a final stable state— this state being the current 
value of each processor as to whether it is "on" or whether it Is "off which represents a legitimate and short 
tour. 

The type of hardware system for efficiently solving the four-city TSP might, for example, consist of a system 
whereby four groups are provided and there are four neuronic processors within each group, such as in FIG. 
2 where the four groups are labeled as I, II, III, and IV. 

Within each group, each neuronic processor Is identified with a numt>er that represents one of the four 
cities, and the arrangement is such that each processor can be in only one of two possible states whereby the 
"on" condition means that that particular city is the one that should take that particular place In the sequences 
of travel routing permutations. 

Not shown in FIG. 2 is the condition where every single one of the 16 processors has a cross-connection 
to every single one of the other processors which would, in effect, represent a combinatorial matrix system. 
These connections between the processors are chosen so as to "encourage" the entire network of processors 
to choose a final stable state, this final stable state being the continuously stable value of the current value of 
each processor, whether "on" or whether "off. 

Since it is known that these system operations will act to minimb^e the global energy (E). this global energy 
(E) can be defined by a particular algorithm which is indbated as Equation A1, shown herein below. 



Where 

E = global system energy. 

N = number of nodes or cities to visit 

Where w is the value, a positive or negative real number, of the connection-weight between the proc- 
essor 1 and the processor]; and X| is the current state (0 or 1) of the 1^^ processor. 

The values of W are chosen so that low energy states correspond to short, legitimate tours. Connections 
between processors within the group are such that high network energy results when more than one processor 
Is "on" In order to discourage those operations involving "non-tours". 

Similarly, connections between processors of two different groups "discourage" the same city from being 
in more than one place in the sequential permutation. 

The system operates on a series of machine cycles. During each machine cycle, each processor consults 
its delay schedule, and, if the comparison between this schedule and the number of the machine cycles that 
have passes Is favorable, will execute Its primary algorithm. This algorithm is indicated herein below as Equa- 
tion A2. and is often called the "decision rule": 
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Where U is the "threshold value" and the symbol "n* is a square matrix of NxN configuration. The ele- 
ments of 'n* are integer samples from a unlfomi distribution designated, for example as 0, 1, 2 .... M. These 
reflect a "fixed delay" of ng time-steps between the processing units i and j. 

The quantity (t-np 6t) is the state of the neuron (1 or 0) at some time in the past 
5 The current existing state of the network would be defined by a vector x as follows: 

Vector X equals (Xi,X2, X3, ...x, ...Xn). [Eq, A3] 
RG. 3 is schematic representation of a typical group of four processors showing how each processor is 
interconnected to and influences each one of the other processors in that group. The square box units which 
connect between each of the processors in the group of four are called nodal weight and dday units. The "plus" 
10 signs indicate positive values of the connection weight -delay value Wg and the "minus" symbols indicate neg- 
ative values of Wq. 

The negative values discourage any two processors from being "on" at the same time. The square box 
units 36 of FIG. 3 will constitute f bced delays which could be built from shift registers, for example, and the 
processors NP1, NP2, NP3, NP4 might be implemented as simple adder circuits. 
15 Additionally, as illustrated in FIG. 3, a processor from another grouping (NP5) is interconnected to each 

and every one of the four processors of the previous group, NP1 , NP2, NP3, NP4. This is to illustrate and give 
some Indication that every one of the processors in the "first group" would be interconnected to each and every 
one of the four processors in the "second group", and likewise the same would hold true for the third and fourth 
groups of processors, as was indicated in regard to FIG. 2. 
20 Referring to the Equation A2 above, it should be indicated that Xj(t) would represent the state (0 or 1) con- 

dition of the i"* processor during the "machine cyde" occurring at any given particular time "t". 

Symbol N would represent the number of processors operating in the system, and the symbol ng would 
represent the square of the number of processors, that is to say N^. 

25 DESCRIPTION OF PREFERRED EMBODIMENT 

The special purpose neurocomputer system of the present disclosure is shown in a preferred embodiment 
thereof in FIG. 1. 

The basic architecture involves a series of processors designated as nodal processors which are identical 
30 in nature and which have been designated as processors 10, 20, 30,...i...N. Since the number of processors 
is a variable, the designation "N" is used to signify the total number of nodal processors In the system and any 
selected intermediate in the system may be designated as processor "i*. For certain purposes, the series of 
Nodal Processors are referred to as NP^. NP2,... NPm. 

As will be seen in FIG. 1, the overall system may be observed in terms of "Nodes" whereby each "node" 
35 consists of a set of intercooperating units. For example, the first node may be conceived as involving the nodal 
processor 10 together with the units designated as the "nodal weight and delay memory" 36, the schedule 
memory 46, the decision algorithm 56, each of which provides inputs to the nodal processor 10. Additionally, 
the first "Node" would also include a temporary memory latch 15 which receives outputs from the nodal proc- 
essor 10. Likewise, each subsequent Node has Its own Memory Latch as ISj, 153....15n. 
40 In FIG. 1, a typical "Node" is shown as NODE 2 with heavy broken lines to show the nodal unit. 

Then similarly, each of the series of other nodes concatenated into the system include the same type of 
units (36n, 46n, 56n) which characterize every other node. 

During each operating cycle, a nodal processor such as 1 0 or 20, etc., consults its schedule unit 46 (462. 
etc.) and the schedule unit informs It as to how many machine cycles it should wait before it evaluates the de- 
45 cislon rule and outputs its result. 

The decision rule, previously cited as Equation A2, is given as follows: 

N 

50 X (t) = 1 if I w X (t-n. 6t) > 

N 

x.(.t) = 0 if^I^ w. . X, (t-n. .6t) < Eq 

Since it is observed that each neuron waits a number of small time steps, or "epochs" (6 t=one machine 
cycle), before evaluating its input and deciding to change its current output state, the Equation A2 is the "de- 
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cision" to change its output state and is determined according to the parameter values in Equation A2. 

The nodal processor, such as 1 0, 20, etc.. will perform the computation of Equation A2 and output the result 
to the network menwry 18. This transfer to the network memory is provided via bus 14, temporary memory 
latch 15, and bus 17. Since a processor's output is simply a 1 or a 0, all of these buses are one bit wide. 

5 The computation of Equation A2 by the nodal processor requires input that represents the output of pre- 

vious computations of other processors. This data is provided in a multiplexed fashion on bus 19m. All of the 
data stored in the network memory, unit 18, will be presented to eacn processor in a multiplexed fashion in 
one machine cyde. Input from bus 11 , from the processor-controller 70 informs the processor 1 0, etc.. which 
previous state is currently being presented on bus 11. This information consists of an 8-bit integer, with a '0" 

10 meaning the current state of the network is being presented, a "1 " meaning the state one machine cyde in the 
past, etc. In conjunction with the delay information from the nodal weight and delay memory, 36, this data from 
the controller 70 allows the processor 10, 20, etc., to determine the past state of any other processor. For ex- 
ample, if ni2=3, from the weight and delay memory 36, then, when data from the controller 70 indicates that 
the state of the network as of 3 machine cydes ago is currently presenting itself on bus 19ml, processor 10 

15 will record the value, 1 or a 0, presented on the second line of that bus. In other words, as the history of the 
system is presented on bus 19ml, ng provides an index informing thef** processor which value to utilize from 
the line on bus 19ml. 

TTie computation of Equation A2 also requires input that represents the "connection weight* between any 
processor i and any processor j. This information is also contained in the weight and delay memory 36 in the 

20 form of an anray, FIG. 6. Each entry in this array, Wy, represents the "amount" of "force" that one particular 
nodal processor is to exert upon another. Large positive values tend to encourage two processors to be in the 
same state; large negative values tend to encourage opposite states.- 

As discussed previously, the nodal weightand delay memory 36 contains the information for the connec- 

tion weight array, w.and the fixed delay array, n. n is an N x N array configured similar to w (see FIG. 6) where 

25 each entry nij represents the number of machine cydes that the output of unit i is to be delayed before its output 
is utilized by unit j. Each entry in w will be stored as a 16-bit signed Integer, and each entry in n. an 8-bit un- 
signed integer. 

The delay schedule memory 46, contains information telling the processor 10 how many machine cydes 
to wait before it evaluates the decision rule (Equation A2). This information consists of a series of integers. 
30 For example, a typical series might look like the following: 
2,1 .3,2,2,2.1 ,3.3,2.3.3,2.4.3.1 ,4.3,2,2,3,3,4,4,4.3,2.4.4.4, ... 

The total length of series will be less than 2.000 entries. The numbers in the series are chosen randomly from 
a discrete uniform distribution whose upper bound is constantly increasing. By this we mean that initially these 
integers might be samples from [0.1.2]. and finally, samples from [0,1.2,3,4,5,6,7,8,9.10]. 
35 Note that the delayed information contained in the array "n" is fundamentally different that the infonmation 

stored in the delay schedule memory (shown as 46 in FIG. 1). The information in the delay schedule memory 
represents random delays, and is responsible for the asynchronous operation of the computer. As stated above, 
this delay tells the processor how many machine cydes to wait before it evaluates the decision rule (Equation 
A2), Whereas the information in entry "ny" in array n tells the processor that when it does evaluate the decision 
40 rule, it is to use the state of processor i "ny" machine cycles in the past In short, the delay schedule tells "when" 
the decision rule is to be evaluated, and the array n, and w. contained in the nodal weight and weight-delay 
memory 36m tells the processor "how" to evaluate that rule. 

The decision algorithm memory 56, shown in FIG. 1, contains the algorithm, in appropriate machine lan- 
guage, for nodal processor, unit 10 in FIG. 1. for the computation of the decision rule. Equation A2. 
45 As seen in FIG. 1, the network memory 18 provides output data of n bits on the "state history bus" des- 

ignated as 18mx. These output bits (n bits) are fed to the multiplexor 16 for distribution on output bus 19 to 
various of the processors via distribution buses such as 19^1 an 19m2. ".19mi—19mn- The content arrangement 
of History Memory 18 is shown in FIG. 4. 

The processor-controller 70 infomis the nodal processor 10 (and also processors 20, 30,...N) via bus 11 
so as to which one of the particular past states is currently being presented on the "state history bus" 18mx. FIG. 
4 indicates how each nodal processor is recorded as being "on" or "off during any given machine cyde. 

As will be seen in FIG. 1, a processor-controller 70 monitors the nodal system network and also provides 
an output through the I/O controller 80 which may present a display to the terminal 85. The terminal 85 may 
be a cathode-ray tube output display or it may be merely a digital printout of selected information. 
55 The processor-controller 70 may be any general purpose processor or may even be a unit such as the 

currently available types of personal computers. 

The processor-controller 70 is used to continuously examine the current state of the network by scanning 
the network memory 1 8. The network memory 1 8 contains a history of the past states of each of the processors 
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10. 20, 30,„.N, as seen in FIG. 4 and Table I. 

The object of the comparison of these past states is in order to see whether or not the 'state condition" of 

each of the nodal processors 10. 20. 30. ...N has finally settled down to an unchanging setof data states which 

would Indicate that the network has reached a point of ''stability". 
5 As long as changes keep occurring in the status states of each of the processors, it ^ understood that 

non-optimum solutions are still being provided, and it is only when the optimum or dose-to-optimum solution 

has been found that the status data In the network memory 18 will indicate that the status states of each of 

the processors have settled down into a semipermanent or permanent set of data which remains unchanging 

as various possibilities (different tour paths) are being explored. 
10 Thus, a problem ts deemed solved or "optimized" when stability of the data status states is achieved, or 

alternatively, some predetermined number of cycles has passed and the most probable optimum solution is 

at hand. 

Stability of the "status states" is achieved when the comparison of successive states shows that the state 
bits are no longer changing from cyde to cyde and thus it is probable that a point of stability or optimization 
15 has occurred in the network. 

The processor-controller 70 will continuously provide information to each of the nodal processors 10, 20, 
30....N about which particular state of the overall global system is cunrently being presented on the state bus 
18„«. 

The processor-controller 70 operates to initialize the computer by downloading data on bus 11 to each of 
20 the nodal processors 10, 20. 30,...N. This downloaded data on bus 11 would include connection weights and 
delays to the nodal weight and delay memory 36, the random delay schedule to the delay schedule memory 
46. and the dedsion algorithm for the decision algorithm memory 56. In addition, the processor-controller will 
initialize the network history memory 18 via bus 18c. This initial data will set the memory with random values, 
1 orO. 

25 Since most current processors have I imitations on the numberofinputdata lines allowable for input signals, 

the number of nodes in the network will most likely exceed the number of data input lines coming into each 
nodal processor 1 0, 20, 30, etc. Thus, the data, as it is presented to each nodal processor, must be multiplexed 
through the multiplexor 1 6. 

In FIG. 1. the multiplexor 16 is seen receiving the output data of n bits from the history network memory 
30 18 via the state history bus 1 S^c- 

The network history memory 1 8 of FIG. 4 can be thought of as a two-dimensional N x P anray where the 
entries in each row represent the state of each processor (the first entry conresponds to the state of first proc- 
essor, the second entry the state of the second, and so on) at some time in the past. The first row. for exanriple 

is the current state jc(t)=(xi(t).X2(t) Xn(t). The second row represents the state of each processor one machine 

35 cyde in the past. x(t)=Xi(t- 6t). X2(t-6t)...., Xn(t-6t), and the last row, the state p machines cycles in the past 
x(t)=x, (t-P6t), X2(t-P5t) x„(t-P6t). 

The history data from the network memory 18 is presented to the multiplexor 16 and to the processor- 
controller 70 on a row-by-row basis. All rows are presented in sequence in each machine cycle. 

During each machine cyde, the current state of each one of the nodal processors 10, 20. 30.... N is read 
40 into the network memory 18 from each of the n latches which constitute the series of memory latches 15,. 
152, 153.. .15n shown in FIG. 1. 

In FIG. 1. the temporary state memory latch 15 is shown receiving input on bus 14 from the nodal processor 
10 and also receiving data on bus 12. An output bus 17 conveys the state information from the latch 15 over 
to the network memory 18. 

45 Since each of the nodal processors 10, 20. 30,...N waits a certain number of machine cydes before eval- 

uating (according to the decision rule of Equation A2) its input and deciding whether it snould cnange state, 
its "current" state must be preserved during these dormant periods. Thus, the output of the latch 15 is presented 
to the network history memory 18 during each machine cyde. 

Thus, each of the "nodes" in the system will have its own temporary memory latch such as 15, 15,. 152, 
50 153,...15,.,..15n, and each of the nodal processors 10. 20,...N. will be presenting its current state, for each ma- 
chine cyde. to the network memory which can then store and carry a history of the state condition of each of 
the nodal processors for each of the previous machine cycles induding the current machine cyde. 

In the work of Hinton. Sejnowski and Ackley, on the Boltzmann machine, there was noted that delays in 
the system will "mimic" synaptic noise of the chemical synapse. These researchers and others have modeled 
55 the output function of a neuron as a "sigmoid-shaped", cumulative distribution-indicating the probability of 
maintained firing versus the input stimuli signals. 

In the Boltzmann machine, neurons are modeled as stochastic units whose output is a Boltzmann distrib- 
ution. In this distribution, "temperature" is a parameter which governs the amount of randomness, and conse- 
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quently the shape. 

The system network disclosed herein has similarities with both Hopf ield's two-state model, and also to 
the Boltzmann machine. Hopfield had demonstrated that a symmetrically connected networtcof binary neurons 
would find a "local minimum" of an extrinsic quantity, which he identified as the system's "energy". He showed 

5 that this model can function as an error-correcting content addressable memory. In his later work, the two- 
state model was dropped for a non-linear analog model. The "smoothness" of the analog model was shown to 
accentuate the identification of "good" solutions to optimization problems. 

In the Boltzmann machine, low energy states are achieved by "simulated annealing", a method of utiilzing 
"noise" to escape the local minima (partially optlmun solutions) . By starting with the "temperature" parameter 

10 relatively "high" and then slowly lowering it. the probability of locating a stable state with low energy was sig- 
nificantly enhanced. This process, called sinmjiated annealing, was modeled since it emulates the attainment 
of low energy quantum states in metals by slow cooling. 

The presently disclosed delay model system network resembles the Boltzmann machine in that delays do 
seem actually to mimic "noise", and noise allows the escape ivom local minima. However, unlike the Boltzmann 

15 machine, the present system provides randomness, and the variability of that randomness is provided as a 
function of time, and these factors appear to be an intrinsic property of the presently disclosed delay model 
system network. 

The presently disclosed neuronic system network is similar to Hopf ield's two-state model. The network 
consists of N "neurons" where each neuron has tv/o states which are indicated as "0" (not firing) or "1", (firing 
20 at a maximum rate). 

The output of the i^ neuron is connected to the input of the j*^ neuron with a "weight" designated as a Wy. 

As will be seen in FIG. 6, the nodal weight-delay memory 36 of FIG. 1 Is seen to provide a relationship 
table whereby the weight-delay time relationship between any two individual neuronic processors in.tbe net- 
work Is provided for and shown in FIG. 6. The matrix w is symmetrk:, that is to say, the output of neuron i in 
25 relationship to neuron j has the same weight as the neuron j has to the neuron i. Since no neuron may synapse 
on itself, there is seen a set of diagonal zeroes to indicate this in FIG. 6. 

This system network of neurons (processors) is asynchronous in the sense that each neuron waits a ran- 
dom, integral numt>er of small time steps, called "epochs" (St), before evaluating its input and deciding to 
change its current stats. This number is chosen from a uniform distribution, [1, 2, 3,...R] where R is the max- 
30 imum number of epochs a neuron waits before it "evaluates" its current status. The decision to change the 
output state is made according to the "decision rule" which is often called the McCulloch-Pitts decision rule 
which was discussed in the Bulletin of Mathematical Biophysics , Vol. 5 1943 in an article entitled "A Logical 
Calculus of the Ideas Imminent in Nervous Activity" by W. S. McCulloch and Walter Pitts. This decision rule 
is shown accordingly to be represented by the hereinbelow equation marked Equation B1: 



N 

x.(t) = 1 if^i^ w.. X. (t-n.^6t) > E^, and 

fEq. Bll 

N 

X- (t) = 0 if I w X. ( t-n. .6t) < 
' j = l J 



45 where E© is the threshold and n is a square matrix of NxN. The elements of n are integer samples from a uniform 
distribution [0, 1 , 2, ...P]. They reflect a "fixed" delay, time of ny time steps between the neuron unit I and the 
neuron unit j. P represents the Maximum number of past states of the system that are to be retained. 
The quantity Xj (t-nySt) is the "state" of the neuron (1 = on or 0 = off) at some time in the past 
As previously stated in Equation A3, the current state of the network may be described by the vector x 
50 equal to (x^, X2. X3....XN). 

In the system network described herein, several sets of simulations were performed. For example, in the 
first set of simulations, the parameters were set up as follows: 

N = 25, M = 25, R = 3, and Eo = 0 
The elements of w^ (see FIG. 6) were set up with real valued samples from the uniform distributk>n [minus 
55 the square root of N, and then the square root of N], such that the symbol wy is equal to wji, and Wh is equal to 
zero. 

During these simulations, with random initial starting points, it was seen that "stable points" were almost 
always found. Stable points were identified only after the network returned the same global state vector for 
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M epochs. The time elapsed before a stable state was found varied widely from a minimum of 50 epochs, and 
in one run the system surged for over 1,000 epochs before the run was terminated without convergence. The 
system appears to move through configuration space randomly until some small islands of stability are 
reached. These islands involved two to three neurons whose state was unchanged for at least M epochs. When 
this condition occurred, the system usually continued with more and more neurons becoming "frozen" and with 
a relatively rapid convergence to a stable state. 

This t>ehavior, though rather unexpected, may, however, be subject to some qualitative explanation. It is 
useful to think of the model neurons (processors) that obey the McCulloch-Pitts decision rule as making de- 
cisions based on the update information that reached them through their inputs. 

However, when "delays** are present the neurons are making decisions based on *'past states" of the sys- 
tem. Since the state of the system is usually constantly changing, there is a "nonzero" probability that with old 
information, a neuron will err in its decision. The neuron will thus not Increase its firing rate, when it normally 
would have if Its Input information had been properly current in time. 

The longer a neuron goes without changing state, the higherthe probability of its transmitting current (pres- 
ent) information to those neurons that are connected to it As the state of more and more neurons becomes 
"fixed", the remaining neurons will be found to utilize a higher percentage of "undelayed" Inputs. In effect, it 
would appear that the decision rule is becoming successively more deterministic toward f beating on a final sol- 
ution. 

For a second set of simulations that were tried, it was sought to measure the neurons' output distribution 
at various levels of activity. The motivation was to detemiine how often a neuron 'made a mistake" as a result 
of receiving old information. It was expected that when activity was high, (a lot of neurons evaluating their input 
per each epoch) that the errors resulting would be high. And likewise, when the activity was low, (only a few 
neurons evaluating their input per epoch) that the enrors would be low. 

The first set of simulations had ishown that the probability of a neuron's changing state actually "decreased" 
as the network converged. Consequently, the noise-like effects of delays had to be separated from the ten- 
dency of the network to progress toward a stable point. This necessitated the introduction of another global 
parameter "p" which was identified as the "probability" of a neuron's changing state. 

In the second set of simulations, it was arranged that all parameters remain the same except R, which 
was allowed to vary, and N, which was set to 100. Neurons evaluated their inputs according to two decision 
rules: (i) the first utilized delay times; (ii) the second decision rule did not utilize delay times so that the symbol 
nij was equal to zero for all sets of ij. 

Recordings were taken when the results of the two rules disagreed. A histogram that measured the number 
of times a disagreement between the two rules occurred was plotted versus a weighted sum of the input. E. 
Unlike the first set of simulations, state vectors were not updated according to the results of the decision rules 
to inhibit convergence. Rather, the decision to change a neuron's state was determined by sampling a uniform 
distribution between zero and one. If the result was less than "p" (probability of a neuron's changing state), 
the neuron's state was flipped. Statistics were gathered but not until at least 25 epochs had elapsed, in order 
to insure that enough state history was maintained to observe the true effects of the delays. 

It was found that the recorded histograms, such as that shown in FIG. 7. closely matched the cumulative 
output distribution which was used in other probabilistic models. Thus these histograms will follow a pattern 
which is indicated by the shown hereinbelow Equation B2. 

'^<"-"°--^ = exp[(-1/TXE-Eo)l.1 
where E is the weighted sum of the input. The parameter Eq shifts the curve horizontally, while T governs the 
shape. For a small value of T, the curve resembles a step function: for large values of T, the curve resembles 
a stretched-out "S". 

T can be considered the measure of the level of noise of "temperature" of the system. Neurons whose out- 
put function obey Equation B2 with T equal to zero are called "deterministic" 

Figures 8 and 9 show the resulting histograms which fit with Equation B2. In FIG. 8, the statistics were 
recorded for 500 epochs. The p equals 0.1. and P equal 3. and T equals 23 as computed from the fit. 

In FIG. 9. the parameters were p = 0.01 , R = 3. and T = 7.7. Similar large shifts in T were found with changes 
in R. 

R represents the maximum number of epochs that a neuron waits before it evaluates its current status. 

Additranal histograms were also computed with nonrandom connection matrices, "w" was loaded with 
"weight-delay times" appropriate for a neural solutton of the five-city Travelling Salesman Problem as devel- 
oped by Hopfield and Tank in 1985 and 1986. In this case, the sigmoid shape was maintained but the curve 
was shifted horizontally. That this curve narrows with low activity, and as the network converges on a stable 
point, would suggest that with an appropriate choice of activity, the system might be made to "cool" slowly and 
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consequently yield "good* solutions to optimization problems. Such a system could be designated as exhibiting 
"auto-annealin9^ 

As a result of the developments of a neuron-simulated system network, it was seen that the two-state mod- 
el type neurons with connections delays have an intrinsic sigmoid-shaped output distritHJtion similar to that 
5 used in other probabilistic models. In spite of this random or stochastic component the simulations showed 
that, after identifying stable isolated neurons, the system usually proceeded to find a stable state. Samples 
of output distribution which were obtained showed that the distribution narrowed— indicating that the system 
was becoming deterministic, or "cooling"— as the network evolved toward a stable state. 

The amount of noise in the network, or "temperature", was also dependent on the mean firing rate. It is 
10 thus considered that a combination of these two effects provided a natural way for a delayed system to min- 
imize the system's "energy". 

There has thus been described a preferred embodiment of a special purpose neurocomputer system for 
solving optimization problems. However, other embodiments and variations may still operate on the concepts 
disclosed herein and may be defined by the following claims. 

15 

Claims 

1. A network of binary processors operable to seek network minimum energy states, comprising: 
20 (1a) a plurality of N processors (10, 20...N) wherein each said processor (N) is either "on" = 1 or "ofT 

= 0 during each network machine cyde, said 1 or 0 constituting status information Xi for the i-th proc- 
essor; 

(lb) a plurality of cross-connectfon means (19mn)for.connecting eachx)neof said K processors to each 
other one of said N processors (10, 20,...N); 

25 (1c) weight-delay means (36„) in each of said cross-connection means (19^0) to carry said status in- 

formation between any two counterpart processors (i and j) as to whether i and j are "on" = 1 or "ofr 
= 0 and to carry a weighted influence signal Wg which tends to inhibit by means of a negative value its 
counterpart processor (i or j) from being "on" (= 1) or encourages by means of a positive value its coun- 
terpart processor to go "on", said positive or negative value of W|j being subject to a fixed delay tinrte 

30 ng in machine cydes before its influence Wg Is effective on the said counterpart processor, according 

to the status defining equation A2: 



N 

x-(t) = 1 if 1 w X. (t-n..5t) >U 

1 i=l ^3 1 ij 

N 

= 0 i £ E w, . X . ( t-n. . 6t:) < U 

i=l ij 1 13 - 



where Xj(t) represents the status (0 or 1) of the i-th processor at time t, 
Wy the weight value between processor i and processor j, 
ny an integer reflecting a number of fixed delay times. 
45 6t represents one machine cyde. 

U an operator-selected threshold value; 

(1d) a network history memory (18) for storing the processor status, of 1 = "on" and 0 = "ofr, for each 
processor for each past number "P" of machine cydes such that each processor may access present 
or past status information to implement a dedsion algorithm means (56n) for the evaluation of said sta- 
50 tus defining equation A2, after which said weight-delay means (36n) becomes operative; 

(1e) processor-controller means (70) for examining the status, in said network history memory (18), 
of each processor over a past number of "P" machine cydes in order to evaluate the stability of the no- 
change condition for "P" machine cycles to indicate when the network of processors has reached a sta- 
ble, non-changing state, whereby network energy according to the equation A1 

55 
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J N N 
^ = 2 [ -ij X, X. 

has reached a minimum; 

i) said weight-deiay means (36n) including: 

(1ca) a weight influence unit (wg) which Is predetermlnedly set according to predefined criteria to 
digital negative values of "inhibit" and/or positive of "encourage", between a processor i and its coun- 
terpart j, to provide Inputs for said decision algorithm means (56n), which will establish which proc- 
essor will be "on" and which will be "off during each machine cyde; 

(1cb) a delay unit which provides integer values, as betv/een any processor I and Its counterpart j, 
of time delay, n^ in machine cycles, before the positive or negative influence of Wq can take effect 
such that when processor] evaluates said status defining equation A2, it uses the status that proc- 
essor I had ng machine cydes In the past; 
(ii) said decision algorithm means (56n) for establishing whether each said processor (10, 20.. .N) will 
have an "off status (= 0) or an "on" status (=1) for any given machine cyde. according to said status 
defining equation A2, said means induding: 

(ita) delay schedule means (46n) for delaying the execution of said status defining equation A2 eval- 
uation for a predetermined number of machine cydes, according to random delay values nQ, thus ef- 
fecting an asynchronous operation of the network. 

2. The network of Claim 1 which includes: 

(2a) multiplexor means for conveying the output data from said network history memory to each 
of said N binary processors for each machine cyde. 

3. The network of Claim 2 wherein each of said processor means includes: 

(3a) a decision rule memory unit for storing a decision rule algorithm, said dedsion rule memory 
unit induding: 

(3a1) said decision rule algorithm for enabling said binary processor to evaluate Information 
received from said network history memory via said multiplexor means in order to determine the output 
state of said binary processor. 

4. The network of Claim 2 or 3 wherein each said processor means includes: 

(4a) a temporary memory latch for storing the status (0 or 1) of the said binary processorand trans- 
mitting the said status to said network history memory for each machine cyde. 

5. The system network of one of the preceding Claims wherein said network Indudes; 

(5a) an I/O controller to receive information data from said processor-controller which indicates said 
stable non-changing network state and then conveys said data to a display means; 
(5b) display means for receiving information data from said I/O controller for display to a human oper- 
ator. 



Patentanspruche 

1. Netzwerk aus binaren Prozessoren zum Suchen minimaler Netzwerkenergiezustande, mit 

(1a) einer Mehrzahl von N Prozessoren (10. 20.. .N), bei welcher jeder Prozessor (N) wahrend jedes 
Netzwerkmaschinenzyklus entweder "an" = 1 oder "aus" = 0 ist, wobei 1 oder 0 eine Statusinformation 
X) fur den i-ten Prozessor bilden; 

(lb) einer Mehrzahl von Querverbindungseinrichtungen (19mn) zum Verbinden jedes einzelnen der N 
Prozessoren mit jedem anderen der N Prozessoren (10, 20,. ..N); 

(1c) einer Gewichtungsverzogerungseinrichtung (36^) in jeder der Querverbindungseinrichtungen 
(19mn) zum Transportieren der Statusinformation zwischen zwei komplementaren Prozessoren (i und 
j), ob i und j "an" = 1 oder "aus" = 0 sind, und zum Transportieren eines gewichteten Einflulisignales 
Wq. welches durch einen negativen Wert seinen komplementaren Prozessor (i oder j) daran hindert, "an" 
(= 1) 2u sein, oder durch einen p)osltiven Wert seinen komplementaren Prozessor dat>ei unterstutzt, 
"an" zu gehen, wobei der positive oder negative Wert von Wg einer festen Verzogerungszeit ny in Ma- 
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schinenzyklen unterworfen wird. bevorsein Einflua Wg auf den komplementaren Prozessor wirkt, ge- 
ma& der statusbeschreibenden Gleichung A2: 

wobei X[(t) den Status (0 oder 1) des t-ten Prozessor zum Zeitpunkt t reprasentlert, 
Wg den Gewichtungswertzwischen Prozessor i und Prozessor j, 

Hfj eine ganze Zahl, die erne Anzahl festgelegter Verzdgerungszeiten wiederspiegelt 5t reprasentlert 
einen Maschinenzyklus, U einen bedienerselektierten Schwetlwert; 

(1d) einem Netzwerkprotokollspeicher (18) zum Speichern des Prozessorstatus von 1 = "an" und 0 = 
"aus" fur jeden Prozessor und f ur jede vergangene Anzahl "P" von MaschinenzyWen, so daR jeder Pro- 
zessor auf gegenwartige Oder vergangene Statusinfonnnationen zugreifen kann, um eine Entschei- 
dungsalgorithmuseinrichtung (56n) zur Bewertung der statusbeschreibenden Gleichung A2zu imple- 
mentieren, nach welcher die Gewichtungsverzogerungseinrichtung (36n) wirksam wird; 
(1e) Prozessorsteuerungseinrichtungen (70) zum Prufen des Status jedes Prozessors in dem Netz-. . 
werkprotokollspeicher (18) uber eine vergangene Anzahl von "P" Maschinenzyklen, um die Stabilitat 
des unveranderten Zustandes fur "P" Maschinenzyklen zu bewerten, um anzuzeigen, wann das Pro- 
zessor-Netzwerk einen stabilen, sich nicht andernden Zustand enreicht hat, wodurch die Netzwerk- 
energie gemaB der Gleichung A1 

, N N 

ein Minimum erreicht hat; 

i) wobei die Gewichtungsverzogerungseinrichtung (36n) beinhaltet 

(1ca) eine GewichtungseinfluBeinheit (wy), welche vorbestimmt gema& vordefinierten Kriterien auf 
digitale negative Werte von "verhindern" und/oder positive von "unterstutzen"zwischen einem Pro- 
zessor i und seinem Komplen>ent j eingestellt ist, um Eingaben fur die Entscheidungsalgorithmus- 
einrichtung (56n) bereitzustellen. welche festlegt, welcher Prozessor wahrend jedes Maschinenzy- 
klus "an" und welcher "aus" sein wird; 

(1cb) eine Verzogerungseinheit, welche ganzzahlige Werte der Zeitverzogemng ny in Maschinen- 
zyklen zwischen jedem Prozessor 1 und seinem Komplement j bereitsteltt, bevor der positive oder 
negative EinfluR von Wy Wirkung zeigen kann, so daB, wenn Prozessor j die statust>estimmende 
Gieichung A2 auswertet er den Status verwendet, den Prozessor i n^ Maschinenzyklen in der Ver- 
gangenheit hatte; 

ii) wobei die Entscheidungsalgortthmuseinrichtung (56n) zum Festlegen, ob jeder Prozessor (10, 
20.,.N) einen "aus"-Status {= 0) oder"an"-Status (= 1)furjeden vorgegebenen Maschinenzyklus gemaR 
der statusbestimmenden Gleichung A2 auf welsen wird, beinhaltet: 

iia) eine Verzogerungstabelle (46„) zum Verzogern der Ausf uhrung der Bewertung der status- 
bestimmenden Gleichung A2 fur eine vorbestimmte Anzahl von Maschinenzyklen gemaR zufalligen 
Verzogerungswerten ntj , um somit eine asynchrone Wirkungsweise des Netzwerkes zu verwirkiichen. 

Netzwerk nach Anspruch 1, welches beinhaltet: 

(2a) Multiplex-Elnrichtungen zum Befordern der Ausgangsdaten des Netzwerkprotokollspeichers 
zu jedem der N binaren Prozessoren fur jeden Maschinenzyklus. 

Netzwerk nach Anspruch 2, bei welchem jeder Prozessor beinhaltet 

(3a) eine Entscheidungsregelspeichereinheit zum Speichern eines Entscheidungsregelalgorith- 
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mus, wobei die Entscheidungsregelspetchereinhert beinhaftet 

(3a1)den Entscheidungsregelalgorithmus zum Freigeben des binaren Pprozessors zum Be- 
werten dervon dem Netzwerkprotokollspeicheruberdie Multiplexeinrichtungen empfangenden Informa- 
tion, UPD den Ausgangszustand des binaren Prozessors zu bestimmen. 

4. Netzwerk nach Anspruch 2 oder 3. bei welchem der Prozessor beinhaltet 

{4a) einen temporaren Verriegelungsspeicher zum Speichern des Status (0 oder 1) des Binarpro- 
zessors und Obertragen des Static an den Netzwerkprotokollspeicher fur jeden Maschinenzyklus. 

5. Netzwerksystem nach einem der vorstehenden Anspruche, wobei das Netzwerk beinhaltet 

(5a) eine E/A-Steuerung zum Empfangen von Informationsdaten von der Prozessorsteuerung, wel- 
che den stabilen, sich nicht andernden Netzwerkzustand anzeigt und dann die Daten zu einer Anzeige- 
einrtchtung bef5rdert; 

(5b) eine Anzeigeeinrichtung zum Empfangen von Infornnationsdaten von der E/A-Steuerung zum 
Anzeigen fur einen menschlichen Bediener. 




Revendications 

1. Roseau de processeurs binaires pouvant etre mis en oeuvre pour rechercher des 6tats d'6nergie minimale 
de r^seau, comprenant : 

(1a) une plurality de N processeurs (1 0, 20, N) dans laquelle cheque dit processeur (N) est soit "pas- 
sant" = 1 ou "bloquant" = 0 pendant cheque cyde de machine de r^seau, ledit 1 ou 0 constituant une 
information d'6tat X| pour le i*™ processeur ; 

(lb) une plurality de moyens de connexion crois6e (IQ^n) pour relier chaque processeur desd its N pro- 
cesseurs d chaque autre processeur desdits N processeurs (10, 20 N) ; 

(1c) un nroyen de retard-poids (36„) dans chacun desdits moyens de connexion crols^e (19m„) pour 
porter ladite information d'6tat en tre deux quelconques processeurs pendants (i et j) quant d savoir si 
! et j sont "passants" = 1 ou "bloquants" = 0 et pour porter un signal d'infiuence pond6r6 wg qui tend d 
inhlber, au ntoyen d'une valeur negative, son processeur pendant (i ou j) d*§tre "passant" (= 1) ou qui 
encourage, au moyen d'une valeur positive, son processeur pendant k devenir •passant", ladite valeur 
positive ou n6gative de Wy 6tant soumise ^ un temps de retard f ix6 Py dans les cycles de machine avant 
que son influence wy ne soit effective sur ledit processeur pendant, selon r6tat d6f inissant T^quation 
A2: 



xi(t) = 1 si Z v/ij Xi (t - nij 5t) > U 
i = l 



= 0 si X Wij Xi (t - ni-j 5t} < U 
i=l 

ou x,(t) repr6sente I'^tat (0 ou 1) du i*"»« processeur d I'lnstant t, 
Wy est la valeur de poids entre le processeur I et le processeur j. 
njj est un entier ref letant un certain nombre de temps de retard fixe, 
6t repr6sente un cycle de machine, 

U repr6sente une valeur de seuil s6lectionn6e par un op6rateur ; 
(1d) une m6moire d'historique de r^seau (18) pour stocker I'^tat du processeur, 1 = "passant" et 0 = 
"bloquant". pour chaque processeur pour chaque nombre pass6 "P" de cycles de machine de sorte que 
chaque processeur peut acc6der aux informations d*6tat present ou pass6 pour r6aliser un moyen d'al- 
gorithme de d6cision (56n) pour r<§vaIuation dudit 6tat d6f inissant I'^quation A2, apr6s quo! ledit moyen 
de retard-poids (36n) devient op^rationnel ; 

(1e) un moyen formant unit6 de commande de processeur (70) pour examiner T^tat, dans ladite m6- 
moire d'historique de r6seau (18), de chaque processeur sur un certain nombre pass6 de "P" cycles 
de machine af in d*6vafuer la stability de la condition de non-changement pour "P" cycles de machine 
pour indiquer quand le r^seau de processeurs a atteint un 6tat stable, non changeant. de sorte que 
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I'^nergie de r^seau, selon I'^quation A1 : 




Wij Xi Xj 



a attaint une valeur minimale ; 
(i) I edit moyen de retard- poids (SSJ comprenant : 

(1ca) une unrt^ d'lnfluence de poids (wq) qui est positionn^e. de maniere pr6d^tennnin6e selon des 
entires pr^d^f Ints, d des valeurs num^riques n^attves d'^lnhibition" et/ou ^ des valeurs positives 
d'"encouragement", entre un processeur i et son pendant j, pour realiser des entries pour ledit 
moyen d'algorithme de decision (56n). lequel va ^tablir quel processeur va etre "passant" et lequel 
va etre "bloquant" pendant chaque cycle de machine ; 

(1 cb) une unit^ de retard qui donne des valeurs enti^res, comma entre tout processeur I et son pen- 
dant j, de retard de temps, ny dans les cycles de machine, avant que Tint luence positive ou negative 
de Wjj ne puisse prendre effet de sorte que lorsque le processeur] lvalue ledit 6tat dSfinissant I'^qua- 
tion A2, il utilise T^tat que le processeur i avait ny cycles de machine dans le pass^ ; 
(ii} ledit moyen d'algorittime de decision (56^) pour ^tablir si chaque dit processeur (10, 20, .... N) va 
avoir un 6tat "bloquant" (= 0) ou un 6tat "passant" (= 1) pour tout cyde de machine donn6. selon ledit 
^tat d^f inissant T^quation A2, tedit moyen comprenant : 

(tia) un moyen d'ordonnancement de retard (46n) pour retarder ['execution de ladite Evaluation 
- d'etat d^f inissant I'Equation A2 pendant.un nombre pr6d6termin§ de cydes de machine, selon des va- 
leurs de retard aI6atoires ny, effectuant ainsi un fonctionnement asynchrone du r6seau. 

Roseau selon la revendication 1. comprenant : 

(2a) un moyen multtplexeur pour transporter les donn^es de sortie de ladite m^moire d'historique 
de r^seau vers chacun desdits N processeurs binares pour chaque cycle de machine. 

Roseau selon la revendication 2. dans lequel chacun desdits moyens processeur comprend : 

(3a) une unitE de m^moire de rdgle de decision pour stocker un algorithme de r^gle de decision, 

ladite unitE de m^moire de r^gle de decision comprenant : 

(3a1) ledit algorithme de rdgie de decision pour permettre audit processeur binaire d'evaluer des 

informations regues depuis ladite m^moire d'historique de reseau par TintermEdiaire dudit moyen multi- 

plexeur af in de determiner TEtat de sortie dudit processeur binaire. 

Reseau selon la revendication 2 ou 3, dans lequel chacun desdits moyens processeur comprend : 

(4a) un circuit de memorisation temporaire pour stocker r6tat (0 ou 1) dudit processeur binaire et 
pour transmettre ledit 6tat ^ ladite m6moire d'historique de reseau pour chaque cycle de machine. 

Reseau de systeme selon Tune des revendications precedentes, dans lequel ledit reseau comprend : 
(5a) une unit6 de commande d*E/S (I/O) pour recevoir des donn^es d'information en provenance de 
ladite unite de commande de processeur. qui indique ledit etat stable non changeant de r6seau et en- 
suite qui transporte lesdites donn6es vers un moyen d'affichage ; 

(5b) un moyen d'affichage pour recevoir des donnees d'information en provenance de ladite unite de 
commande d'E/S (I/O) pour les aff icher pour un operateur humain. 
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